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We study the observability for a lepton flavor-changing decay of a Higgs boson h fir at hadron 
coUiders. Flavor-changing couplings of a Higgs boson exist at tree level in models with multiple 
Higgs doublets. The hfir coupling is particularly motivated by the favorable interpretation of 
— i^T oscillation. We find that at the Tevatron Run H the unique yur signature could serve as the 
Higgs discovery channel, surpassing expectations for Higgs boson searches in the SM and in a large 
parameter region of the MSSM. The sensitivity will be greatly improved at the LHC, beyond the 
coverage at a muon collider Higgs factory. 
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The standard model (SM) of electroweak interactions 
and many of its extensions generically predict the exis- 
tence of Higgs bosons. Detecting Higgs bosons and study- 
ing their properties in future coUider experiments would 
provide crucial information for the mechanism of elec- 
troweak symmetry breaking and hopefully fermion flavor 
physics as well. These have been the most prominent 
issues in contemporary particle physics. 

The upgraded Fermilab Tevatron will start its mis- 
sion next year with c. m. energy ^/s = 2 TeV and an 
annual luminosity L « 2 fb~^ per detector (Run Ha). 
Ultimately, one would hope to reach an integrated lu- 
minosity of L w 15 — 30 fb^^ (Run lib). In terms 
of the search for the SM Higgs boson (h), the most 
promising processes beyond the LEP2 reach would be 
electroweak gauge boson-Higgs associated production 0] 
pp Wh, Zh. The leptonic decays of W, Z provide a 
good trigger and h bb may be reconstructible with ad- 
equate 5-tagging and bb mass resolution, allowing a Higgs 
boson reach of nih ^ 120 — 130 GeV For a heavier 
Higgs boson rrih « 2Mw, the leading production channel 
via gluon fusion gg h and the relatively clean decay 
mode h WW* tvlv may be useful in digging out a 
weak Higgs boson signal j^] . It is believed that a SM-like 
Higgs boson may be observable up to a mass of about 180 
GeV at a 3ct statistical level for L « 25 fb~^ Q. In the 
minimal supersymmetric extension of the standard model 
(MSSM), the mass of the lightest CP-even Higgs boson 
is bounded by mt < 130 GeV g. When the CP-odd 
Higgs state (A) of the MSSM is heavy ruA > 2Mz, the 
lightest Higgs boson has SM-like properties and the con- 
clusion for a light SM Higgs boson search remains valid in 
a large parameter region of the MSSM. The only excep- 
tion is when tua ~ 0{Mz) and tan/3 (ratio of the Higgs 
vacuum expectation values) is large, where the produc- 
tion of bbh, bbA is enhanced by tan^ /3 and h,A^ bb, tt 
may be accessible At the CERN Large Hadron Col- 
lider (LHC) with ^/s = 14 TeV and L 100 - 300 fb~\ 
one expects to fully cover the range of theoretical interest 
mil ^ 1 TeV for the SM Higgs boson, or to discover at 



least one of the MSSM Higgs bosons §]. 

The Higgs sector is the least constrained in theories 
beyond the SM. It is thus prudent to keep an open mind 
when studying Higgs physics phenomenologically and ex- 
perimentally. A particularly important question about 
the Higgs sector is its role in fermion flavor dynamics, 
i.e., the generation of fermion masses and flavor mixings. 
There have been attempts to explain flavor mixings by a 
generalized Higgs sector with multiple Higgs doublets. It 
is argued that the fermion flavor mixing structure due 
to the Higgs coupling at tree level can be of the form, 

h ipiifjj , (1) 

where i,j are generation indices and v w 246 GeV is 
the Higgs vacuum expectation value. Kij is a product 
of the model parameter and the neutral Higgs mix- 
ing cos a [Q . Although they are free parameters without 
a priori knowledge of a more fundamental theory. Ay is 
naturally order of unity from a model-building point of 
view and cos a — I corresponds to no Higgs mixing. Such 
Higgs-fermion couplings would yield flavor-changing neu- 
tral currents, and therefore lead to rich phenomenology 
|p|-|lT|. However, transitions involving the light genera- 
tions are naturally suppressed and the largest couplings 
occur between the third and second generations. 

In this Letter we explore the lepton flavor-changing 
coupling of a Higgs boson. This is particularly mo- 
tivated by the favorable interpretation for i^^t — Vt fla- 
vor oscillation from recent atmospheric neutrino experi- 
ments [p^ . If a large mixing between i/^ and i^^ exists 
as indicated by the Super-K experiment [p^ , then it will 
necessarily lead to the decay h fir. The branching 
fraction depends on the particular model of the Higgs 
sector, which can be parameterized by k^. The cur- 
rent constraints on this coupling from low energy exper- 
iments are rather weak, giving A^,- < 10 derived from 
the muon anomalous magnetic moment Q. Other low 
energy probes are not expected to be sensitive enough to 
reach the natural size A^r ~ C'(l). The potentially in- 
teresting lepton flavor-changing decay modes for a Higgs 



1 



10^ 



0^ 



10" 



10" 



10" 



. . . . . 1 . 


, , , 1 , , , , 1 , , , , 1 


, , , 1 , , , ... 


: 6b 








WW..-' 




~ TT 






bs 











... 1 ... . 


: fJ.r 

■ . . . . 1 . 


... 1 .... 1 .... 1 



100 



no 



150 



160 



120 130 140 

mh [GeV] 

FIG. 1. The Higgs boson decay branching fraction versus 
ruh- The couphng parameters Kij are taken to be one. 



and their search at 
In this work, we 



boson were recently discussed 
a muon coUider (l3| was studied |11 
propose to look for the signal at the upgraded Tevatron 
and the LHC. The leading production mechanism for a 
neutral Higgs boson through giuon fusion is 

pp[p) ^ ggX -^hX^ iitX. (2) 

We find that due to the unique flavor-changing signature 
and the distinctive kinematics of the signal final state, the 
Tevatron Run II will have significant sensitivity to such 
a coupling, making this signal a possible Higgs discovery 
channel for mu « 100 - 140 GeV if ~ 0(1). At the 
LHC, the sensitivity is substantially improved leading to 
a probe for the coupling to a level of k^t ~ 0.15 and 
extending the mass coverage to 160 GeV. 

h production and decay at hadron colliders 

The dominant decay mode for a SM-like Higgs boson 
is /i ^ 66 for rail < 130 GeV and h — s- WW* for a heavier 
mass. The partial decay width for ft, —> /it is given by 



T{h fir) 



(3) 



Here and henceforth fir = fj, + ^~^t . In com- 
parison to the T+r~ mode in the SM, we have 
r(/i^/xr)/r(ft^Tr) =2K2,(m^/m,). In Fig. 0, we 
show these decay branching fractions versus the Higgs 
boson mass. The /ir mode is plotted assuming = 1, 
for which BR(/i fir) is at the 1% level. For k, 3, 
the /ir mode can be as large as the SM t^t" mode. For 
rail > 140 GeV, this mode dies away quickly due to the 
opening of the large WW* mode. This is the primary 
reason for the limitation to a low Higgs mass (m/j < 140) 
at a muon collider [ pTp^ ]. 

In Fig. 1^ we show the total cross section for gg — > /i as 
well as the final states from the h decay versus rrih at the 
(a) Tevatron and (b) LHC. The production is SM-like as 
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FIG. 2. The Higgs boson production cross-section via 
gluon-fusion versus mh at the (a) Tevatron and (b) LHC. The 
solid curve is for the ^it mode, assuming k^t = 1. The scales 
on the right-hand side give the number of events expected for 
(a) 4 fb"^ at the Tevatron and (b) 10 fb"^ at the LHC. Vari- 
ous subsequent decay modes r"'"r~, WW* and WW* —> ivlu 
are depicted for comparison. 



we take ku = 1 . We normalize our signal cross section to 
include next-to- leading order QCD corrections [Q, and 
use the CTEQ4M distribution functions fl^ . The scales 
on the right-hand side give the number of events expected 
for 4 fb""'^ at the Tevatron (the 2 fb""'^ luminosity at the 
CDF and DO detectors are combined) and 10 fb~^ at the 
LHC. We see that for the range of 110—140 GeV and 
K^r = 1, there may be about 10 — 40 events produced at 
the Tevatron and 100 - 4000 events at the LHC. 

/i — > /ir signal and SM backgrounds 

The signal final state /ir is quite unique: two flavor- 
changing charged leptons back-to-back in the transverse 
plane without much hadronic activity. To estimate the 
observability of the signal in hadron collider environ- 
ments, we consider the r to decay to an electron or 
(at least one charged) hadrons, excluding the mode to a 
muon. We do not require explicit r tagging in the anal- 
ysis. We simulate the detector coverage at the Tevatron 
(LHC) by imposing some "basic cuts" 



> 20 GeV, > 10 GeV, \t]\ < 2 (2.5), 



(4) 



where pt^ (p^ ) is the transverse momentum for the muon 
(charged track and other observable hadrons from r de- 
cay) , and T] is their pseudo-rapidity. We further simulate 
the detector energy resolutions at the Tevatron B 



(5) 



AEj/Ej = 0.8/VEj for hadrons, 
AEe/Ee = 0.2/ for electrons, 
and at the LHC [1 



AEj /Ej = 0.65/ VE^j ® 0.05 for hadrons, 
AEjEe = O.l/V^e © 0.005 for electrons. 



(6) 



The muon is required to be well isolated and we neglect 
the smearing. We finally veto extra jets in the range 
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p{ > 20 GeV, IV'I < 3 (7) 

to maximally preserve the signal kinematics. 

Although the lepton flavor-changing signal is quite 
spectacular, it is not background-free. The leading SM 
backgrounds include the Drell-Yan (DY) process 

PP{P) Z{j*) T+T^ ^IV^Vr T, (8) 

and W^W^ pair production (M^VK) 

pp{p) -> W+W~ M^^M T^r- (9) 

The background processes are calculated with the full SM 
matrix elements at tree level including spin correlations 
of gauge boson decays. QCD corrections as X-factors 
for the total production rates are also taken into account 
1^ . With the basic cuts of Eq. (^), the backgrounds turn 
out to be very large. The results are given by the entries 
under "basic cuts" in Tables || and ^ for the Tevatron 
and LHC, respectively. 

There are several distinctive kinematical features for 
the signal that we can exploit to discriminate it from 
the backgrounds. First, the missing neutrinos from r 
decay are collimated along the charged track since the 
r's are ultra-relativistic. Thus, for the signal, the missing 
transverse momentum (p^*'"*) along the charged track 
direction and is essentially back-to-back with respect to 
the muon ±) « 180°. This is not the case for the 
WW background. Secondly, the muons in the signal are 
stiff ~ 'rrih/'i- as a result of the two-body Higgs decay; 
while the secondary tracks and hadrons from r decay are 
softer. If we define momentum imbalance 

ApT=P^-p^, (10) 

we expect that it would be positive for the signal if the 
momentum measurements were perfect. This variable 
turns out to be very powerful in separating the DY back- 
ground. We now define the "refined cuts" as 

±) > 160°, Apt > 0, > mh/h. (11) 

The most important aspect for the signal observation 
is reconstruction of the Higgs boson mass. This is quite 
feasible for the signal under consideration. This can be 
done with the following steps: (1) define the missing 
transverse momentum p™*^" as the imbalance from the 
observable particles (which is ApT hi Eq. (|l^) for the 
signal case); (2) reconstruct the t transverse momen- 
tum = + p^^^'^ , and the longitudinal component 
pI = pf{l + Pt^'^" /Pt)'j (3) form the pr invariant mass 
"^^T ^ (p^ +p^)^. This mass variable should be sharply 
peaked at m/j for the signal, broadly peaked around Mz 
for the DY background, and rather smooth over a large 
range for the WW background. Indeed, with the proper 
energy smearing, we find the reconstructed Higgs mass 
peak within a 5 GeV range. The results are summarized 
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TABLE I. Signal h ^ and SM background cross sec- 
tions at the 2 TeV Tevatron for = 100 - 140 GeV and 
K^T- = 1 after different stages of kinematical cuts. The signal 
statistical significance S/^/B is presented for 20 fb"'^. 
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tions at the 14 TeV LHC for = 100-160 GeV and k^^ = 1 
after different stages of kinematical cuts. The signal statisti- 
cal significance S/VB is presented for 10 fb~^. 

in Tables | and || for the Tevatron and LHC, respectively. 
The entries under "refined cuts" give the cross sections 
including the cuts of Eq. ( pdj ) . The signal-to-background 
ratio S/B within a 5 GeV window for m^r is shown next. 
The last rows illustrate the statistical significance S/^/B 
for the Tevatron with 20 fb~^ (CDF and DO combined) 
and for the LHC with 10 fb"!. 

Discussion and conclusion 

So far, for our signal discussion, we have chosen the 
coupling parameter as k^t = 1 for illustration. From a 
model-building point of view, it is natural for to be of 
order unity, while the upper bound from low energy con- 
straint is about 10. Generically, the cross section scales 
like Kj^^. We explored to what value of this coupling the 
signal would yield a 3<t evidence statistically near the 
Higgs mass peak. Figure ^ shows k^^ versus ruh at the 
(a) Tevatron and (b) LHC for several luminosities. We 
see that at Run Ha where a luminosity of 4 fb""'^ is ex- 
pected combining CDF and DO data, k^^. ~ 1.2 — 1.8 can 
be reached for m/i < 140 GeV. With a higher luminos- 
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FIG. 3. The value of k^t at which the signal yields a So- 
statistical evidence, versus mh at the (a) Tevatron and (b) 
LHC for several luminosities. 
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FIG. 4. Integrated luminosity needed to reach a 2a (95% 
exclusion), Scr and 5cr signal versus vrih at the (a) Tevatron 
and (b) LHC for K^^r ~ 1. 



ity of 30 fb~^ per detector, one can reach a coupling of 
0.6 — 0.9. At the LHC, the sensitivity is significantly im- 
proved and a signal for k^t '--^ 0.15 would even be observ- 
able with 100 fb^^. Assuming k^,- « 1, the reach could 
go beyond rrih ~ 160 GeV, in contrast to the accessible 
limit rrih < 140 GeV at a muon collider Similarly, 
one can ask how much luminosity is needed to reach a 
certain level of observation. Note that the statistical sig- 



nificance scales like S/y/B ^ kI^^/L. The results are 
summarized in Fig. ^, where a 2a (95% confidence level 
exclusion), 3ct and 5cr signals are illustrated versus mh 
at the (a) Tevatron and (b) LHC for k^,- = 1. Due to 
the large number of signal events near the mh peak at 
the LHC (see Table ||) , the statistical accuracy of deter- 
mining a coupling k^,- ~ C(l) can be at a few percent 
level with only L = 10 fb~^. Note that strictly speaking, 
all the bounds quoted here apply to the product kuh^t- 
We have implicitly assumed ku = 1 throughout. 

In summary, we have studied the observability for a 
lepton flavor-changing decay of a Higgs boson ft, —> /ir at 
the upgraded Tevatron and the LHC. At the Tevatron, 
the unique signature may serve as the Higgs discovery 
channel, yielding a 3cr signal for mh ~ 110 GeV and 
K^r ~ 1.2 with 4 fb^^ (CDF and DO combined), surpass- 



ing expectations for Higgs boson searches in the SM and 
in a large parameter region of the MSSM. The sensitivity 
will be greatly improved at the LHC, probing as small a 
coupling as k^,- 0.15 or determining k^t ~ C'(l) better 
than a few percent accuracy, and extending the reach to 
mh ^ 160 GeV, beyond the coverage at a muon collider. 
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